Gower Rank 

Jan de Leeuw 

Version 0.03, Jun 08, 2016 


Abstract 

In Multidimensional Scaling we sometimes find that stress does not decrease if we 
increase dimensionality. This is explained in this note by using the Gower rank. Some 
examples with small Gower rank are analyzed. 
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Note: This is a working paper which may be expanded/updated frequently. All suggestions 
for improvement are welcome. The directory gih.stat.ucla.edu/gower has a pdf version, the 
complete Rind hie with all code chunks, and the bib hie. 


1 Theory 

In full-dimensional scaling (De Leeuw (1993), De Leeuw, Groenen, and Mair (2016)) or FDS 
we minimize a loss function defined as 

n n 

(1) 

i=l j =1 

over all positive senri-dehnite (psd) matrices C. Here w,j are known non-negative weights 
and Sij are known non-negative dissimilarities. To prevent complete triviality we assume 
Wijdij > 0 for at least one pair i,j. The distances dij(C ) are defined by 
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( 2 ) 


dij (C ) . \Jc-jj -\- Cjj 2cij . 

It follows that cr is convex on the convex cone of psd matrices, and thus all local minimizers 
of the FDS problem are global. Suppose X is the set of all pairs (i, j) with WijSij > 0. Then 
(De Leeuw (1984)) at a local minimum dij(C ) > 0 for all (i,j) G X. Thus a is differentiable 
in a neighborhood of local minima. 

We need some additional definitions. We use the Loewner order, i.e. A > B means that 
A — B is psd. Define the matrices 

Aij [e-i — ej)(e.i — e 3 ) , 

with e; and e 3 unit vectors, i.e. vectors with one element equal to one and the rest equal to 
zero. Note that d t j(C) = J tr AijC. Using the matrices A l3 we define 


(3) 

j=ij=i 
n n r 

A):=EE w a w 

j=\j=\ a ijy^) 

If we assume, for convenience, that YAj=\ = 1, then a(C ) = 1 —2tr B(C)C+ tr VC. 

From convex analysis (Rockafellar (1970), theorem 31.4) the necessary and sufficient conditions 
for C to be a minimizer are 


^>0, 

(5) 

B{C) < V, 

(6) 

tr ( B{C ) - V)C = 0. 

(7) 


In general we do not have uniqueness. Think of the case in which X has only one element, 
say (1, 2). Then any psd matrix C with cn + C 22 — 2ci2 = Sf 2 is a minimizer with stress equal 
to zero. The local minimum is unique if and only if C = 0 is the only psd solution of the 
homogeneous linear system d?-((7) = tr A l3 C = 0 with (i,j) € X. Let us assume that this is 
the case, and that the global minimizer C is of rank p. Then p is called the Gower rank of 
the pair ( W , A). If the minimum is not unique the Gower rank is the minimum rank of the 
minimizers. 

There is a clear analogy with classical MDS, in which the corresponding quantity p* is 
the number of positive eigenvalues of —the doubly-centered matrix of squared 
dissimilarities. In fact, the Gower conjecture is that the Gower rank p is less than or equal to 

p*. 
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If C — XX' is a full rank decomposition, with X of dimensions n x p, then we can write (7) as 
V + B(C)X = X, using superscript + for the Moore-Penrose inverse. Thus X are eigenvectors 
of V + B(C ) with eigenvalues equal to one. Condition (6) says that all eigenvalues of V + B[C) 
are less than or equal to one, and X corresponds with the p largest eigenvalues. 


2 Algorithm 

Multidimensional scaling (MDS) algorithms such as smacof (De Leeuw and Mail' (2009)) 
parametrize the MDS problem in terms of an n x p configuration X. Thus 


»W~EE - <i„(W ) 2 (») 

i=lj =1 


with dij(X) = yj tr A^XX'. An MDS(p) algorithm minimizes a over all n x p configurations. 

An iterative MDS(p) algorithm converges to a configuration with B(X)X = VX, where B(X) 
is dehned by (4) with C = XX'. Thus C = XX' solves (5) and (7), but the eigenvalues of 
V + B{X) can be larger than one, and consequently C does not solve the FDS problem. But 
conversely if the MDS(p) solution X gives a matrix V + B(X) with p eigenvalues equal to one, 
then p is the Gower rank and C = XX' solves the FDS problem. This also implies that X 
gives the global minimum of the MDS(q) problem for all q > p. 


3 First Example 


The following asymmetric matrix is taken from Wang and Chin (2010), table 5. Wong 
and Beasley (1990) constructed artificial data comparing seven hypothetical departments 
in a university in terms of three input variables (number of academic staff, academic staff 
salaries, support staff salaries) and three output variables (number of undergraduates, number 
of postgraduate students, number of resaerch papers). The data are used to construct a 
cross-efficiency matrix, which uses linear programming to compute input and output weights 
that measure efficiency of a department relative to another department. This produces cross 
efficiencies between zero and one, which can be interpreted as similarities, so we actually use 
one minus the data from Wang and Chin (2010). 


## a b 

## A 0.0000 0.0188 
## B 0.0781 0.0000 
## C 0.0000 0.1490 
## D 0.3125 0.0000 
## E 0.0000 0.1539 
## F 0.0000 0.0188 
## G 0.0000 0.0188 


b 

0.2310 

0.2281 

0.0000 

0.2651 

0.3349 

0.2310 

0.2310 


d 

0.3589 

0.2987 

0.5458 

0.1803 

0.5865 

0.3589 

0.3589 


d 

0.0618 

0.1010 

0.5050 

0.2350 

0.0000 

0.0618 

0.0618 


f 

0.0000 

0.0000 

0.0000 

0.0494 

0.0896 

0.0000 

0.0000 


g 

0.0000 

0.0000 

0.7059 

0.0000 

0.0000 

0.0000 

0.0000 
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It was observed by Shahin Ashkiani, PhD candidate at Autonomous University of Barcelona, 
that metric unfolding solutions for this matrix exhibit some curious behaviour (personal 
communication, June 4, 2016). Increasing the dimensionality of the solution does not decrease 
the minimum stress. The MDS(p) solution has the same minimum stress for all p > 2. This 
indicates the Gower rank of these data is two. 

Although metric unfolding can be more efficiently done by smacof Rect (), we use smacof Sym() 
on the 14 x 14 augmented matrix, with the data in the off-diagonal submatrices and with zero 
weights for the diagonal submatrices. This corresponds more directly with the notation we 
use in this paper. We start with a random configuration and iterate until difference between 
successive stresses is less that le-15. 


## 

ndim = 1 

1 

iterations = 

4 

1 

stress = 

0.4693852450 

## 

ndim = 2 

1 

iterations = 

150 

1 

stress = 

0.4067723531 

## 

ndim = 3 

1 

iterations = 

133 

1 

stress = 

0.4067723531 

## 

ndim = 4 

1 

iterations = 

142 

1 

stress = 

0.4067723531 

## 

ndim = 5 

1 

iterations = 

113 

1 

stress = 

0.4067723531 

## 

ndim = 6 

1 

iterations = 

135 

1 

stress = 

0.4067723531 


We can also illustrate this by showing the singular values of the configuration computed by 
smacof SymO. 

## [1] 3.099044 

## [1] 2.574971 1.997667 

## [1] 2.5749717 1.9976662 0.0000001 

## [1] 2.574971 1.997667 0.000000 0.000000 

## [1] 2.5749716 1.9976663 0.0000003 0.0000000 0.0000000 

## [1] 2.574971 1.997667 0.000000 0.000000 0.000000 0.000000 


4 Second Example 


There is a more interesting example in Wang and Chin (2010), analyzing cross efficiences 
computed from data of Tofallis (1997). In this case the decision making units are 14 airlines. 
The input variables are aircraft capacity, operating cost, and non-flight assets. The output 
variables are passenger kilometres and non-passenger revenue. The cross-efficiences are 


## 


a 


b 


c 


d 


e 


f 


g 


h 


i 


j 


k 


1 


## 

A 

0. 

, 13 

0. 

IX) 

X) 

0. 

.38 

0. 

13 

0. 

, 15 

0. 

.53 

0. 

.19 

0. 

,21 

0. 

00 

o 

0. 

X) 

CN 

0. 

, 13 

0. 

.23 

## 

B 

0. 

00 

00 

0. 

.66 

0. 

,95 

0. 

00 

00 

0. 

00 

00 

0. 

.98 

0. 

.75 

0. 

.73 

0. 

CN 

N- 

0. 

.79 

0. 

00 

00 

0. 

o 

00 

## 

C 

0. 

, 12 

0. 

00 
1— ^ 

0. 

.05 

0. 

12 

0. 

, 12 

0. 

.31 

0. 

to 

00 

0. 

.32 

0. 

00 

00 

0. 

,22 

0. 

, 12 

0. 

.19 
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## 

D 

0 

.04 

0 

.57 

0 

GO 

O 

0. 

,04 

0 

.06 

0 

00 

o 

0 

.18 

0. 

,21 

0 

.30 

0. 

19 

0 

.04 

0. 

, 17 

## 

E 

0 

.03 

0 

.63 

0 

.00 

0. 

,03 

0 

.00 

0 

.00 

0 

.23 

0. 

,26 

0 

.22 

0. 

00 

0 

.03 

0. 

,00 

## 

F 

0 

. 12 

0 

.89 

0 

.04 

0. 

, 12 

0 

. 12 

0 

.02 

0 

.34 

0. 

,39 

0 

.49 

0. 

28 

0 

. 12 

0. 

,25 

## 

G 

0 

.08 

0 

.22 

0 

.52 

0. 

,08 

0 

. 12 

0 

.66 

0 

.00 

0. 

,00 

0 

. 16 

0. 

22 

0 

.08 

0. 

,20 

## 

H 

0 

.22 

0 

.39 

0 

.48 

0. 

,22 

0 

.23 

0 

.71 

0 

.15 

0. 

, 14 

0 

. 18 

0. 

25 

0 

.22 

0. 

,24 

## 

I 

0 

.21 

0 

.27 

0 

.49 

0. 

,21 

0 

.21 

0 

.73 

0 

. 12 

0. 

,09 

0 

.05 

0. 

16 

0 

.21 

0. 

,16 

## 

J 

0 

.22 

0 

.36 

0 

.35 

0. 

,22 

0 

. 17 

0 

.64 

0 

.22 

0. 

,21 

0 

.00 

0. 

00 

0 

.22 

0. 

,03 

## 

K 

0 

.00 

0 

.00 

0 

.57 

0. 

,00 

0 

.00 

0 

.56 

0 

.00 

0. 

,00 

0 

.00 

0. 

00 

0 

.00 

0. 

,00 

## 

L 

0 

.05 

0 

.37 

0 

.25 

0. 

,05 

0 

.04 

0 

.56 

0 

.06 

0. 

,06 

0 

.00 

0. 

00 

0 

.05 

0. 

,00 

## 

M 

0 

.00 

0 

.57 

0 

.00 

0. 

,00 

0 

.00 

0 

.54 

0 

.00 

0 . 

.00 

0 

.00 

0. 

02 

0 

.00 

0. 

,00 

## 

N 

0 

.00 

0 

.77 

0 

.00 

0. 

,00 

0 

.00 

0 

.00 

0 

.22 

0 . 

.27 

0 

.35 

0. 

14 

0 

.00 

0. 

. 12 


## m n 
## A 0.13 0.13 

## B 0.83 0.83 

## C 0.12 0.12 

## D 0.04 0.04 

## E 0.03 0.03 

## F 0.12 0.12 

## G 0.08 0.08 

## H 0.22 0.22 

## I 0.21 0.21 

## J 0.22 0.22 

## K 0.00 0.00 

## L 0.05 0.05 

## M 0.00 0.00 

## N 0.00 0.00 

We repeat the same analysis as in the first example. In this case we conclude the Gower rank 
is equal to three. 


## 

ndim = 1 

iterations = 

6 

1 

stress = 

0.4760530537 

## 

ndim = 2 

iterations = 

278 

1 

stress = 

0.2437571250 

## 

ndim = 3 

iterations = 

1523 

1 

stress = 

0.2386715692 

## 

ndim = 4 

iterations = 

1943 

1 

stress = 

0.2386715692 

## 

ndim = 5 

iterations = 

2013 

1 

stress = 

0.2386715692 

## 

ndim = 6 

iterations = 

1862 

1 

stress = 

0.2386715692 


The singular values for the second example are 

## [1] 4.503339 

## [1] 4.153864 2.830820 

## [1] 4.151090 2.632152 1.042914 

## [1] 4.151090 2.632154 1.042909 0.000020 
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## [1] 4.151090 2.632154 1.042909 0.000020 0.000000 

## [1] 4.151090 2.632154 1.042909 0.000020 0.000000 0.000000 

5 Third Example 

The third example was already analyzed in De Leeuw, Groenen, and Mair (2016). The Ekman 
(1954) color data give similarities between 14 colors. We transform to dissimilarities by using 
the transformation (1 — x) 3 , and we repeat the analysis from the previous sections. Note 
that this is not an unfolding example, all weights are equal to one. The Gower rank turns 
out to be two, i.e. for this transformation the two-dimensional smacof solution is the global 
minimizer of stress in p > 2 dimensions. 


## 

ndim = 1 

1 

iterations = 

7 

1 

stress = 

0.5131738031 

## 

ndim = 2 

1 

iterations = 

44 

1 

stress = 

0.1049991045 

## 

ndim = 3 

1 

iterations = 

161 

1 

stress = 

0.1049991045 

## 

ndim = 4 

1 

iterations = 

178 

1 

stress = 

0.1049991045 

## 

ndim = 5 

1 

iterations = 

167 

1 

stress = 

0.1049991045 

## 

ndim = 6 

1 

iterations = 

172 

1 

stress = 

0.1049991045 


The singular values for the third example are 

## [1] 2.188205 

## [1] 2.060186 1.477827 

## [1] 2.0601856 1.4778274 0.0000007 

## [1] 2.0601856 1.4778274 0.0000007 0.0000000 

## [1] 2.0601856 1.4778274 0.0000007 0.0000000 0.0000000 

## [1] 2.0601856 1.4778274 0.0000007 0.0000000 0.0000000 0.0000000 

6 NEWS 

001 06/06/16 

• First release 
002 08/06/16 

• Third Example 
003 08/06/16 
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• Added something on uniqueness 

• Added Gower conjecture 
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